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I.   INTRODUCTION 

The  reliability  of  any  system  composed  of  N  components  is  defined: 

R(T  )  =  Probability  (T  >  T  ) 
o  o 

where  T  denotes  the  time  of  first  system  failure.   A  significant  practical 
problem  is  that  it  is  often  reasonably  easy,  and  comparatively  inexpensive, 
to  determine  information  about  the  reliability  of  separate  component  types, 
denoted  R. (T  ),  i  =  1,2,...,  N  ,  but  fairly  difficult  and  extremely  expen- 
sive to  determine  total  system'  reliability  directly,  since  failure  testing 
often  destroys  the  system. 

In  the  most  common  analytically  treated  case,  statistically  independent 
components  in  series ,  where  each  of  the  component  types  has  an  assumed  ex- 
ponential distribution  of  failures,  i.e., 

-X.T 
W  =  e  X  °»  i  =  1>2>""  N  »  CD 

the  system  reliability  is  given  by 

-XT        N 
R(T  )  -  e    °,  X  -  I     X.  .  (2) 

i=l 

(It  should  be  noted  in  complex  systems  that  these  are  valid  only  when  all 
components  operate  the  same  fraction  of  mission  time.) 

A  statistically  exact  procedure  for  obtaining  upper  bounds  on  X   from 
data  on  component  failure,  here  called  the  LR  procedure,  has  been  developed 
by  Lieberman  and  Ross  (1971).   In  this  procedure  k.   items  of  each  type 
component,  with  individual  failure  times  T..,  i  =  1,2,...,  N  and 
j  =  1,2,...,  k.   are  to  be  observed.   A  time  U  is  defined: 

k, 


(  >  ) 

U  =  min.  *  £  T..   .  (3) 


i.e.,  U  is  the  cumulative  time  at  which  one  first  exhausts  all  the 
components  of  one  type.   The  number  of  each  type  of  component  which  has 
been  used  up  to  this  time  if  given  by: 


n.  =  largest  {j  <  k.    T  T. „  <  U  .  }. 
i     ■       J  —  x  '  L.      11  —    min 


(4) 


1=1 

Then,  if 

N 
K  -  I     n  (5) 

i=l 

(i.e.,  total  components   used),  Lieberman  and  Ross  showed  that  2XU  follows 

2 
the  Xotr  distribution;  hence  upper  bounds  for  X  can  be  estimated. 

A  second  procedure  for  estimating  upper  bounds  on  X  is  the  approximat 

optimum  procedure  for  Type  II  censoring  (fixed  number  of  failures)  developec 

by  Mann  and  Grubbs  (1974) ,  which  approximates  the  optimum  exact 

bound  of  El  Mawaziny  (1965) .   In  this  Mann  and  Grubbs  (hereafter  referred 

to  as  MG)  procedure,  it  is  assumed  that  testing  of  all  components  of  type 

i  commences  simultaneously,  and  continues  until  r.   of  the  k.   original 

have  failed.   (It  is  permissible  that  r.  =  k.)   Then,  if  the  individual 

failure  times  are  ordered  so  that  T . .  <  T. /..,%,   the  total  test  time  for 

the  i   component  type  is  defined, 

ri 

Z.  =  I     T. .  +  (k.  -  r.)T.    .  (6) 

1   .*•-  11     i    i  ir. 

3=1  i 

The  approximate  optimum  upper  bound  for  X,  at  confidence  level  a, 
is  given  by: 


>MG     1  i    v  +  "^  1  m 

X   =  m  >   1 s-  +  I     ,  (7) 

(     9m^    3m   ) 


where  n   is  the  100a    percentile  of  the  standard  normal  distribution, 


and 


and 


N 
m  -  I     C(r±  -  1)/Zi>  +  1/U  ,  (8) 


i-1 

N 


v  =  l     {(r.  -  1)/Z?}  +  1/U2  ,  (9) 

i=l 

with  U  defined  by  (3)  above. 

A  major  drawback  of  the  LR  technique  is  that  in  using  K  as 

defined  by  (4)-(5),  one  "discards"  the  data  on  the 

N 
NL  =  I      {k  -  n  }  (10) 

i=l 

components  which  do  not  fail  by  the  time  U.   While  necessary  for  the 

statistical  validity  of  the  procedure,  this  means  that,  effectively, 

the  LR  procedure  assumes  components  of  each  type  are  tested  sequentially, 

st 
i.e.  testing  of  the   (j+1)    component  of  type  i  commences  with  failure 

of  j    component  of  that  type.   This  is  seldom  the  testing  scheme  in 

practice. 

An  immediate  consequence  of  this  and  (4)  is  that  the  LR  procedure 

is  Data-Order   Dependent,  that  is,  permuting  the  second  subscript  on 

T .  .   (i.e.,  in  essence  permuting  the  order  in  which  the  failures  are 

observed)  can  alter  n.,   and  hence  the  LR  estimated  bound.   This  can 

l 

lead  to  significant  practical  problems,  for  often  failure  data  on  individual 
components  is  gathered  in  simultaneous  (parallel)  testing  of  all  k. 
components,  rather  than  in  sequential  testing.   Thus  there  is  no  preferred 
ordering,  and,  before  using  the  LR  procedure,  the  analyst  is  faced  with 
the  formidable  task  of  deciding  on  the  "best"  way  to  assign  the  T . . . 


It  is  obvious  that  the  MG  procedure  does  not  suffer  these  drawbacks. 
(Note  that  the  data  order  dependence  of  the  LR  method  is  a  mixed  blessing, 
since  it  may  allow  the  analyst  to  construct  several  alternative  bounds  to 
X,   all  of  equal  statistical  validity,  by  simply  permuting  the  data.) 

The  purpose  of  this  paper  is  to  compare  the  LR  and  MG  procedures 
in  terms  of  the  expected  bounds,  and  effects  of  "discarded"  data,  using 
both  analytic  and  simulation  techniques. 


II.   COMPARISON  OF  THE  TWO  METHODS-ANALYTIC  RESULTS 

Immediate  comparison  of  the  LR  and  MG  procedures  is  complicated  by  the 

fact  the  test  schemes  are  conceptually  different.   As  noted  above,  the  deri- 

s  t 
vation  of  the  LR  procedure  presupposes  that  testing  of  the   (j  +  1)    com- 

t-Vi 

ponent  of  any  type  begins  with  the  failure  of  the  j    component  of  that 
type,  and  that  all  component  testing  ceases  at  U  ,  with  the  exhaustion  of 
the  supply  of  one  type  of  component.   Thus,  conceptually,  the  LR  ends  with 
(N  -  1)   components  (one  of  each  of  the  other  types)  still  undergoing  testing, 
and  no  failed  plus  others,  (NL  +  1  -  N)   to  be  exact,  not  even  tested.   By 
contrast,  the  MG  procedure  implicitly  assumes,  for  each  i  ,  testing  of  all 
k.   components  of  type  .  i  starts  simultaneously,  and  terminates  with  a  fail- 
ure of  some  one  of  them.   Thus  before  data  gathered  according  to  the  MG 
procedure  can  be  utilized  to  produce  a  LR  bound,  the  analyst  must  first 
choose  one  (or  more)  a  priori  orderings  of  the  data. 

However,  for  exponential  failure  data,  we  appeal  to  the  constant  failure 
rate  property  to  convert  data  gathered  in  a  LR  test  format  into  MG  format. 
Precisely,  constant  failure  rate  implies  that  irrespective  of  whether  testing 
of  the  i    component  type  terminates  at  a  failure,  the  sufficient  statistics 
are  total  test  time  and  number  of  failures  observed  in  that  time.   Thus,  if 
all  testing  is  halted  at  U  according  to  the  LR  model,  then,  we  shall  apply 
the  MG  procedure  (providing  each  component  type  experienced  at  least  one 
failure)  with: 

Z.  =  U      ) 

1  i  =  1,2,... ,  N  .  (11) 

r.  =  n.  >  1 
l    i     / 

(This  does  mildly  violate  the  MG  assumptions,  in  that  the  n.   is  random,  not 


set  a  priori.   However,  any  effect  sould  be  noticeable  only  in  extremely  small 

M( 


MG 
sample  sizes,  and  will  not  affect  our  basic  conclusion  unless  ?{\,       <  X}   is 


significantly  increased  as  a  result.) 
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From  this,  it  immediately  follows  that 

N 


m  =  I     {(n  -  1)/U>  +  1/U 
i=l 


(12) 


where  K  is  given  by  (5),  and,  similarly, 


N 

I      (n  -  1)  +  1  |  /U  =  (K  +  1  -  N)/U, 
i-1  ) 


v  =  (K  +  1  -  N)/U2  .  (13) 

But,  with  these  values,  (7)  reduces  to 

XMG  ,  (K  +  1  -  N)   j1.__l_+ "«  __    |3.  (14) 


b        U 


(    »«  +  !-»   3Ak  +  1  -  B)  ( 


Observe  (14)  is  a  strictly  increasing  function  of  (K+l-N)  for  U 
fixed  and  n  <  5.65  (which  includes  all  cases  of  practical  interest). 
Thus,  since  N  _>  2, 


_1 j.      na     3 

3/K  -  1  ) 


\  i^ir-  1-9ar^Tr  +  — =  >  (15) 


Since  the  bound  produced  by  the  LR  method  from  this  data  is 

.LR   X2K(a)  n„ 

\    =  — 2if"  ■       "        (16) 


we  see  that,  when  the  MG  procedure  is  applied  to  the  same  data  actually 
used  in  the  LR  procedure,  the  computed  bounds  satisfy, 


(\m/XLR\   <  2(K  -  X)  i  i      1     .    \   )3 


(17) 


In  Table  1,  we  show  values  of  the  right  hand  side  of  (17)  for  representative 
K  and  a.   It  is  obvious  that,  for  all  practical  values  of  K  and  a, 


(«R) 


<  1. 


Thus,  when  both  the  Lieberman-Ross  and  Mann-Grubbs  procedures  are 
used  to  compute  an  upper  bound  to  X ,   if  the  MG  procedure  uses  only 
the  same  failure  times  as  are  actually  used  by  the  LR  procedure,  the  upper 
bound  produced  by  the  Mann  Grubbs  procedure  will  be  smaller  than  that 
produced  by  Lieberman-Ross. 

An  immediate  extension  of  the  above  result  is  that  the  LR  procedure 
may  produce  lower  bounds  than  the  MG  procedure  only  in  those  cases  where 
the  LR  procedure  does  not  utilize  all  the  data  available.   Specifically, 
consider  the  situation  where  the  LR  procedure  utilizes  failure  times  on 
all  but  one  component  of  each  type  except  the  type  which  exhausts  first, 
which  we  assume  is  type  i  =  1  .   Testing  is  assumed  to  have  continued 
on  these  (N  -  1)   components  until  they  also  failed.   Thus: 

nl  =  kl» 
1    L  (18) 


n±   *  kt  -  1,  i  =  2,  ... ,  N, 


and 


Z,  -  u, 
1  (19) 

Z±   >  U,   i  =  2,  ...,  N. 

Let  e  >  0  and  arbitrarily  small,  and  assume  that  each  component  of  type 
i  >  2,  on  test  at  t  =  U,  fails  at  t  =  U  +  e.  Thus,  MG  can  be  applied 
with: 


r±   -  k±       i  =  1,  2,  ...,  N, 
zx  =  U 

Zi(e)  =  U  +  e,   i  =  2 N. 

Since,  for  e  arbitrarily  small,  Z . (e)  <  Z  ,  i  =  2,  . . . ,  N, 

we  are  thus  understating  the  reliability  of  components  of  type  i,  i  >_  2, 

MG 
Hence  we  expecte  the  bound  computed  using  Z . (e)  ,  denoted  X,  (e)   to 


be  conservative,  i.e. 


and,  more  specifically 


,MG  „  ,MG,  . 
Xb  <  Xb  (e) , 


Xb  K.  £ime->o  Xb  (e)  •  (21) 


It  is  easily  shown 


.MG. 


"•^w-fj1-!^!3.  (22> 


with, 

N  N                      N 

K  =  I     (r  -  1)  +  1  -  I  (k  -  1)  +  1  =  k  +  J  (k  -  1) , 

1=1   1  i=l                  1-2 

i.e.   K  is  degree  of  freedom  from  the  LR  procedure  on  this  data.   Thus, 

for  this  data, 

{h  h    '       X, (a)   | 


(23) 

2K' 


K   3^j 


Computations  show  that,  for  .  75  <_  a  <_  .995,  and  2<_k<  120,   the 
righthand  side  of  (23)  does  not  differ  from  unity  by  more  than  0.4%,  except 
k  =  2  and  a  =  .995,  when  it  differs  from  unity  by  0.8%.   (Some  of 


this  may  be  due  to  minor  inaccuracies  in  the  computer  routines  used.) 
Thus,  we  conclude  that  the  Lieberman-Ross  method  can  not  produce  a  signifi- 
cantly lower  upper  bound  on   X   than  the  Mann  Grubbs  method,  unless  individual 
component  failure  times  are  collected  separately,  and  combined  into  pseudo-system 
failures  in  such  a  way  that  the  LR  procedure  assumes  some  of  the  components 
were  never  tested,  i.e.  the  LR  method  must  deliberately  discard  some  of 
the  data  in  order  to  compute  a  better  bound  than  KG. 

Since  both  the  Lieberman-Ross  and  Mann-Grubbs  procedures  estimate 
upper  bounds  to  X,  it  follows  that, unless  the  confidence  level  (a)  is 
sufficiently  low  that  computing  a  false  bound  is  significantly  probable, 
the  method  producing  the  lower  estimated  bound  will  be  superior.  Thus, 
we  look  for  the  LR  method  to  be  superior  to  MG  when  the  ordering  of 
the  data  chosen  for  the  LR  method  is  such  that  some,  probably  a 
significant  fraction,  of  the  available  data  on  failure  times  is  not 
utilized,  i.e.  when  NL,  as  defined  by  (10)  satisfies  NL  >  (N-l) . 


Ill   COMPARISON  OF  THE  TWO  METHODS  -  SIMULATION  RESULTS 

Having  determined  that  the  cases  where  the  Lieberman-Ross  method 
should  be  superior  to  the  Mann-Grubbs  procedure  should  arise  among  those 
where  the  data  ordering  used  for  the  LR  procedure  "discards"  some  of 
the  available  failure  data,  we  proceded  to  investigate  this  relationship 
in  more  detail.  We  felt  that  the  most  useful  measure  would  be  the 
probability  that  the  LR  method  would  produce  a  superior  bound,  as  a 
function  of  the  amount  of  data  unused.   Since  analytic  techniques  were 
not  able  to  shed  much  additional  light  in  this  area,  we  turned  to  simulation 
The  simulation  described  in  [4],  which  allows  generation  of  component 
failure  data  for  up  to  twenty  individual  components  of  up  to  six  different 
types,  was  modified  for  this  purpose.  We  restricted  ourselves  to  the 
two  component  type  (N=2)   case  for  simplicity.   Four  individual  runs, 
each  run  consisting  of  500  iterations,  were  conducted  for  each  of  the 
combinations  of  confidence  levels,  components  and  failure  rates  shown  in 
Table  2.   In  each  iteration,  independent  exponential  failure  times  were 

generated  for  all  the  components  specified  of  each  type,  and  the  MG 

MG 
procedure  applied  to  these  data,  to  produce  a  bound,   A,  ,   at  the 

specified  confidence  limits.   Then  the  data  were  ordered  into  a  single 

LR  pseudo-failure  sequence.   The  manner  in  which  individual  component 

failures  were  generated  served  to  insure  this  was  a  suitably  random 

ordering  of  the  failure  times.   The  LR  procedure  was  applied  to  this 

LR 
sequence,  to  produce  the  bound  A   .   The  two  bounds  were  compared, 

and  the  LR  procedure  deemed  to  produce  the  superior  bound  if  either  of 

the  following  held: 


10 


(l)    x  <  xb    <  \b    , 

(2)  Xf  <X£A^R, 

(3)  Xf  <X£R<X. 

In  each  iteration,  this  result  was  compared-   with  the  percentage  of 
available  (i.e.  generated)  data  not  used  by  the  LR  procedure: 

N  N 

PL  =  {  I      (k  -  n  )}/{  I     k.}  ,  (24) 

i=l  i=l 

i+f 

where  f  is  the  index  of  the  component  type  which  was  first  exhausted 
i.e.  nf  ■  kf  . 

At  the  completion  of  each  500  iteration  run,  the  percentage  of 
iterations  where  the  data  ordering  had  produced  a  superior  bound  by  the 
LR  method  was  computed,  plotted,  and  tabulated  as  a  function  of  the 
percentage  of  lost  data,  PL.  The  results,  for  each  run,  were  given  in 
a  function: 


I 


The  percentage  of  data  orderings 


TOTR(J)  =  {      ""V05"""  *  lL   VM  J  Wh"e    <25) 

the  LR  method  produced  a  superior 


bound. 

(Note  TOTR(J)  was  set  to  -1.0  if  no  data  orderings  with 
.05(J-1)  <  PL  <_  .05J  were  observed  on  that  run.)   The  plotted  distribution 
of  values  of  TOTR(J)   for  runs  at  a  =   .95   and  a  =  .80  are  shown  at 
Figures  1  and  2  respectively.   Since  TOTR(J)   can  be  regarded  as  an 
estimate  to  the  probability  that  the  LR  method  produces  a  superior 
bound,  given  that  between  5(J-1)%  and  5J%  of  the  data  was  ignored, 


11 


it  is  obvious  that  only  if  the  analyst  is  willing  to  order  the  data 
in  such  a  way  that  80%  or  more  of  the  available  data  is  not  used,  will 
the  probability  that  the  LR  bound  is  superior  likely  exceed  one  half. 
(The  reason  for  the  superiority  of  the  LR  procedure  in  some  cases  when 
little  data  was  discarded  is  that,  in  these  cases,  the  MG  procedure 
produced  a  false  bound,  i.e.   X,   <  X  .) 

From  Figures  1  and  2  it  appears  the  LR  procedure  performs  slightly 

better  vis-a-vis  the  MG  procedure  at  a  =  .80  than  at  a  -   .95.  Essential] 

MG 
this  is  the  result  of  the  probability  that  X,   <  X  increasing  as  a 

decreases. 

Based  on  our  observation  that  an  analyst  would  have  to  order  LR 

data  in  such  a  way  that  the  amount  of  discarded  data  exceeded  80%  of 

that  available  before  the  LR  could  be  reasonably  expected  to  out  perform 

MG,  the  best  strategy  to  do  this  would  be  to  pick  the  T..   in  decreasing 


order,    i.e. 


Tia+i)  <Tu  •     i  =  1- 2 N  • 


This  is  precisely  equivalent  to  discarding,  as  much  as  possible,  the 

least  reliable  samples  of  each  component  type.   This  is  not,  philosophicall 

a  particularly  attractive  strategy. 
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IV.    SUMMARY  AND  CONCLUSIONS 

In  this  paper  we  have  examined  two  different  methods  for  computing 
bounds  on  series  system  reliability,  the  statistically  exact  procedure  of 
Lieberman  and  Ross,  and  the  approximately  optimum  procedure  of  Mann  and 
Grubbs.   Our  analysis  showed  that,  when  used  with  the  same  data,  the  MG 
procedure  would  always  produce  a  lower  upper  bound  on  A   than  the  LR  procedure 
Thus,  the  MG  procedure  seems  preferable,  especially  at  high  confidence  levels, 

since  the  probability  of  computing  a  false  bound  is  correspondingly  small 
there.   Further  analysis  and  simulation  results  showed  that  the  LR 
procedure  could  be  reasonably  expected  to  produce  a  superior  bound  to  MG 
only  where  the  failure  times  were  ordered  such  that  the  LR  procedure 
essentially  "discarded"  most  of  the  data,  and  this  is  usually  equivalent 
to  retaining  samples  of  only  the  most  reliable  components. 

Thus,  it  appears,  at  present,  that  using  the  LR  procedure  would  be 
preferable  to  using  the  MG  procedure  only  in  very  special  circumstances. 
The  only  qualification  we  should  note  is  that  we  have  not  fully  utilized 
the  LR  procedure's  ability  to  usually  produce  multiple  bounds  from  the 
same  data.   This  may  be  exploited  in  some  way  to  make  the  LR  procedure 
more  competitive,  however  we  are  not  certain  of  that  at  this  time. 
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TABLE  2 
A.   Sample  Sizes  Used  in  Simulation 


2 


2  2 

2  3 

2  4 

2  5 

3  3 
3  4 
3  5 
5  5 
5  10 

10  10 

1 


B.   Ratio  of  Failure  Rates  Used  in  Simulation 

(X1/X2)  =  10.00,  5.00,  2.50,  1.25,  1.00,  0.500,  0.250,  0.125,  0.100 


For  the  two  component  case,   (A  A~)   is  only  rate  needed. 
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LIST  OF  FIGURES 

Figure  1    Computed  values  of  TOTR(J) ,   as  defined  by  (25),  for  a  =  .95 
Figure  2    Computed  values  of  TOTR(J) ,  as  defined  by  (25),  for  a  =  .80 
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